Adaptive motion instability detection in video

ABSTRACT

One or more apparatus and method for adaptively detecting motion instability in video. In embodiments, video stabilization is predicated on adaptive detection of motion instability. Adaptive motion instability detection may entail determining an initial motion instability state associated with a plurality of video frames. Subsequent transitions of the instability state may be detected by comparing a first level of instability associated with a first plurality of the frames to a second level of instability associated with a second plurality of the frames. Image stabilization of received video frames may be toggled first based on the initial instability state, and thereafter based on detected changes in the instability state. Output video frames, which may be stabilized or non-stabilized, may then be stored to a memory. In certain embodiments, video motion instability is scored based on a probability distribution of video frame motion jitter values.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application61/829,417, filed on May 31, 2013 and titled “ADAPTIVE SHAKINESSDETECTION FOR VIDEO STABILIZATION.”

BACKGROUND

A graphics engine, graphics processing unit (GPU), or visual processingunit (VPU), is a specialized electronic circuit designed to rapidlymanipulate and alter memory to accelerate the creation of images in aframe buffer typically intended for output to a display. GPUs may befound in embedded systems, mobile phones, tablets, notebook computers,high performance computation (HPC) servers, and game consoles. Inaddition to manipulating computer graphics, a highly parallelarchitecture also enables a GPU to more generally perform processing oflarge blocks of video stream and image data in parallel.

Video cameras are now ubiquitous in mobile electronic media devices.Video cameras are available in wearable form factors (e.g., videocapture earpieces, video capture headsets, video capture eyeglasses,etc.), as well as embedded within smartphones, tablet computers, andnotebook computers, etc. In many circumstances, these devices captureimage sequences (i.e., video) under non-ideal conditions and theacquisition equipment is highly constrained by the mobile form factor.For example, in situations such as filming from a moving vehicle orusing an unsteady hand, many videos show a high degree of undesirablemotion instability or motion jitter (i.e., shakiness). Even videosacquired in less severe conditions often show undesirable shaking.

Video stabilization aims at removing undesired shaky motion from videos.Although some capture devices provide mechanical image stabilization,image processing techniques are often employed in the alternative, or inaddition to, mechanical stabilization. Such image processing techniquestypically involve calculating image motion and performing some form offrame warping to counter the calculated image motion. Stabilization viaimage processing can however severely degrade the visual quality of aninput video, particularly where the input video does not suffersignificant motion instability, or where the stabilization algorithmrelies on assumptions ill-suited to the input video.

Image processing-based motion stabilization might be advantageouslyenabled only as needed based on a motion sensor, such as gyroscopeand/or accelerometer. However, such a front-end solution can be readilyapplied only by the capture device, which often may not have the neededprocessing resources and/or secondary sensor capability.

BRIEF DESCRIPTION OF THE DRAWINGS

The material described herein is illustrated by way of example and notby way of limitation in the accompanying figures. For simplicity andclarity of illustration, elements illustrated in the figures are notnecessarily drawn to scale. For example, the dimensions of some elementsmay be exaggerated relative to other elements for clarity. Further,where considered appropriate, reference labels have been repeated amongthe figures to indicate corresponding or analogous elements. In thefigures:

FIG. 1A is a functional block diagram of a video transmissionarchitecture, in accordance with an embodiment;

FIG. 1B is a functional block diagram of a video stabilization system,which may be employed within the video transmission architecturedepicted in FIG. 1A, in accordance with an embodiment;

FIG. 2 is a flow diagram illustrating a method of adaptive motioninstability detection for video stabilization, in accordance with anembodiment;

FIG. 3 is a flow diagram illustrating a method of adaptively togglingvideo stabilization based on motion instability states, in accordancewith an embodiment;

FIG. 4 is a flow diagram illustrating a method of determining an initialvideo motion instability state and monitoring for transitions in theinstability state, in accordance with an embodiment;

FIG. 5 is a data flow diagram illustrating processing of video todetermine an initial video motion instability state, in accordance withan embodiment;

FIG. 6 is a state diagram illustrating state transitions monitored by anadaptive motion instability detection system, in accordance with anembodiment;

FIG. 7 is a data flow diagram illustrating processing of video tomonitor a transition in video motion instability state, in accordancewith an embodiment;

FIG. 8 is a diagram of an exemplary system, in accordance with anembodiment; and

FIG. 9 is a diagram of an exemplary system, arranged in accordance withan embodiment.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

One or more embodiments are described with reference to the enclosedfigures. While specific configurations and arrangements are depicted anddiscussed in detail, it should be understood that this is done forillustrative purposes only. Persons skilled in the relevant art willrecognize that other configurations and arrangements are possiblewithout departing from the spirit and scope of the description. It willbe apparent to those skilled in the relevant art that techniques and/orarrangements described herein may be employed in a variety of othersystems and applications beyond what is described in detail herein.

Reference is made in the following detailed description to theaccompanying drawings, which form a part hereof and illustrate exemplaryembodiments. Further, it is to be understood that other embodiments maybe utilized and structural and/or logical changes may be made withoutdeparting from the scope of claimed subject matter. Therefore, thefollowing detailed description is not to be taken in a limiting senseand the scope of claimed subject matter is defined solely by theappended claims and their equivalents.

In the following description, numerous details are set forth, however,it will be apparent to one skilled in the art, that embodiments may bepracticed without these specific details. Well-known methods and devicesare shown in block diagram form, rather than in detail, to avoidobscuring more significant aspects. References throughout thisspecification to “an embodiment” or “one embodiment” mean that aparticular feature, structure, function, or characteristic described inconnection with the embodiment is included in at least one embodiment.Thus, the appearances of the phrase “in an embodiment” or “in oneembodiment” in various places throughout this specification are notnecessarily referring to the same embodiment. Furthermore, theparticular features, structures, functions, or characteristics describedin the context of an embodiment may be combined in any suitable mannerin one or more embodiments. For example, a first embodiment may becombined with a second embodiment anywhere the particular features,structures, functions, or characteristics associated with the twoembodiments are not mutually exclusive.

As used in the description of the exemplary embodiments and the appendedclaims, the singular forms “a”, “an” and “the” are intended to includethe plural forms as well, unless the context clearly indicatesotherwise. It will also be understood that the term “and/or” as usedherein refers to and encompasses any and all possible combinations ofone or more of the associated listed items.

As used throughout the description, and in the claims, a list of itemsjoined by the term “at least one of” or “one or more of” can mean anycombination of the listed terms. For example, the phrase “at least oneof A, B or C” can mean A; B; C; A and B; A and C; B and C; or A, B andC.

The terms “coupled” and “connected,” along with their derivatives, maybe used herein to describe functional or structural relationshipsbetween components. It should be understood that these terms are notintended as synonyms for each other. Rather, in particular embodiments,“connected” may be used to indicate that two or more elements are indirect physical, optical, or electrical contact with each other.“Coupled” may be used to indicated that two or more elements are ineither direct or indirect (with other intervening elements between them)physical, optical, or electrical contact with each other, and/or thatthe two or more elements co-operate or interact with each other (e.g.,as in a cause an effect relationship).

Some portions of the detailed descriptions provide herein are presentedin terms of algorithms and symbolic representations of operations ondata bits within a computer memory. Unless specifically statedotherwise, as apparent from the following discussion, it is appreciatedthat throughout the description, discussions utilizing terms such as“calculating,” “computing,” “determining” “estimating” “storing”“collecting” “displaying,” “receiving,” “consolidating,” “generating,”“updating,” or the like, refer to the action and processes of a computersystem, or similar electronic computing device, that manipulates andtransforms data represented as physical (electronic) quantities withinthe computer system's circuitry including registers and memories intoother data similarly represented as physical quantities within thecomputer system memories or registers or other such information storage,transmission or display devices.

While the following description sets forth embodiments that may bemanifested in architectures such system-on-a-chip (SoC) architectures orGPU architectures for example, implementation of the techniques and/orarrangements described herein are not restricted to particulararchitectures and/or computing systems and may be implemented by anyarchitecture and/or computing system for similar purposes. Variousarchitectures employing, for example, multiple integrated circuit (IC)chips and/or packages, and/or various computing devices and/or consumerelectronic (CE) devices such as set-top boxes, smartphones, etc., mayimplement the techniques and/or arrangements described herein. Further,while the following description may set forth numerous specific detailssuch as logic implementations, types and interrelationships of systemcomponents, logic partitioning/integration choices, etc., claimedsubject matter may be practiced without such specific details.Furthermore, some material such as, for example, control structures andfull software instruction sequences, may not be shown in detail in ordernot to obscure the material disclosed herein.

Certain portions of the material disclosed herein are implemented inhardware, for example as logic circuitry in a graphics processor.Certain other portions may be implemented in hardware, firmware,software, or any combination thereof. At least some of the materialdisclosed herein may also be implemented as instructions stored on amachine-readable medium, which may be read and executed by one or moreprocessors (graphics processors and/or central processors). Amachine-readable medium may include any medium and/or mechanism forstoring or transmitting information in a form readable by a machine(e.g., a computing device). For example, a machine-readable medium mayinclude read only memory (ROM); random access memory (RAM); magneticdisk storage media; optical storage media; flash memory devices;electrical, optical, acoustical, or other similarly non-transitory,tangible media.

One or more system, apparatus, method, and computer readable media aredescribed below for adaptive motion instability detection in video. Inthe exemplary embodiments described in detail below, imageprocessing-based video stabilization is predicated on the adaptivemotion instability detection. However, the adaptive motion instabilitydetection techniques and systems described herein may also be utilizedin other application spaces. Adaptive motion instability detection mayentail determining an initial motion instability state associated with aplurality of video frames. Subsequent transitions of the instabilitystate may then be detected by comparing a first level of instabilityassociated with a first plurality of the frames to a second level ofinstability associated with a second plurality of the frames. Motionstabilization processing of received video frames may be toggled firstbased on the initial instability state, and thereafter based on detectedchanges in the instability state. Output video frames, which may bestabilized or non-stabilized, may then be stored to a memory. In certainembodiments, video motion instability is scored based on a probabilitydistribution of video frame motion jitter values.

FIG. 1A is a functional block diagram of video transmission architecture101, in accordance with an embodiment. Architecture 101 includes a videocapture device 103 and a video display device 104 either coupleddirectly together, or coupled together through video cloud service 125.Capture device 103 includes an adaptive video stabilization system 115A.Adaptive video stabilization system 115A receives as an input a videostream output by video capture pipeline 105. Stabilized video output byvideo stabilization system 115A is passed to encoder 116. Encoded videois then stored in video storage 120 for subsequent playback, and/or fortransmission to cloud video service 125. Cloud video service 125 maythen process the encoded video, for example passing the video throughvideo stabilization system 115B. Display device 104 receives encodedvideo, which is passed through decoder 126. Image quality of a decodedvideo stream is improved by processing in de-noise/image enhancer 127.Adaptive video stabilization system 115C receives an input video streamoutput by de-noise/image enhancer 127. Stabilized video is then outputto video memory and/or display 128.

In embodiments, one or more of adaptive video stabilization systems115A, 115B, or 115C include processor logic to detect the presence ofvideo motion instability based solely on image processing. In furtherembodiments, one or more of video stabilization systems 115A, 115B, or115C include video processor logic to apply image stabilizationprocesses selectively to video in response to detecting such motioninstability. Motion instability detection and stabilization may beperformed as part of a post-production process, for example by eitheradaptive video stabilization system 115A or 115B. Advantageouslyhowever, instability detection and stabilization as described herein maybe performed substantially in real time, for example by adaptive videostabilization system 115C, as video display device 104 performs videoplay back. For embodiments where more than one of adaptive videostabilization systems 115A, 115B, and 115C perform image-basedinstability detection and stabilization, various associated detectionand stabilization parameters may be specified for each, allowing formultiple levels of stabilization particular to guidelines chosen bymanagers of the respective systems.

FIG. 1B is a functional block diagram of a video stabilization system118, in accordance with an embodiment. Adaptive video stabilizationsystem 118 may be employed as a component of any of video stabilizationsystems 115A, 115B, or 115C. Adaptive video stabilization system 118receives video from video input pipeline 105, the components of whichmay vary with the context in which adaptive video stabilization system118 is implemented. Video is input into image-based video stabilizationsubsystem 117, which in the exemplary embodiment further includes motionestimator 130 and motion compensator 190. Motion estimator 130 includeslogic (circuitry) to determine an estimate of global motion in the inputvideo stream, for example across sequential frames based on regionsperceived as background, etc. Motion compensator 190 includes logic toreduce or eliminate motion instability from the video stream, forexample by applying one or more frame warping algorithms. Generally, anymotion estimation and any motion compensation may be performed by videostabilization subsystem 117. For embodiments herein however,compensation of estimated motion is conditioned on, or predicated upon,a detection of motion instability. As further shown in FIG. 1B forexample, output from motion estimator 130 is analyzed by adaptive motioninstability detector 140, with motion compensation performed by motioncompensator 190 predicated on a detection of motion instability withinthe input video. As such, motion compensation is made selective tomotion-instable video frames. For any given video input, videostabilization subsystem 117 may pass either, or both, motion stabilizedand non-stabilized video portions to video output pipeline 195. Videooutput pipeline 195 may include memory (e.g., one or more frame buffers)and one or more displays (e.g., LCD, etc.).

In exemplary embodiments, video motion instability detection is adaptiveincluding both an initial instability state detector 150 and aninstability state transition detector 170. Initial instability statedetector 150 includes logic to determine a level of motion instabilityin an input video and then to initiate motion compensation, or not,based on an initial instability threshold. Motion compensator 190 is toapply video/image stabilization algorithm(s) to video frames (or not)until such compensation is toggled by instability state transitiondetector 170 in response detecting a change in video motion instability.In embodiments, as described further below, instability state transitiondetector 170 includes logic to compare a first level of instabilityassociated with a first plurality of video frames with a second level ofinstability associated with a second plurality of frames. An independentthreshold may be applied to this comparison. In exemplary embodiments,both initial instability state detector 150 and instability statetransition detector 170 are coupled to jitter value buffer 145 andinclude logic to utilize jitter values from buffer 145 in theirrespective functions, as is described further below. With this adaptivevideo stabilization architecture, motion compensation may beadvantageously turned “on” promptly (e.g., within a few tens of frames)and also turned “on” and “off” as needed throughout the duration of avideo, which may comprise many thousands of frames. The ability to adaptto changes in video motion instability substantially in real time mayadvantageously reduce the possibility that motion compensator 190performs unnecessarily and/or to the detriment of image quality.

FIG. 2 is a flow diagram illustrating a method 201 of adaptive motioninstability detection for video stabilization, in accordance with anembodiment. In an embodiment, method 201 is performed by adaptive videostabilization system 118. Method 201 begins with receiving input videoframes at operation 205. Global motion is estimated at operation 230.Global motion estimation entails finding the dominant motion vectorbetween adjacent video frames (e.g., between a pair of consecutiveframes). Many algorithms are known for analyzing video to arrive at anestimate of dominant motion between frames, and embodiments describedescribed herein are not limited in this respect. At operation 235,camera trajectory smoothing is performed. For smoothing operation 235, asmoother camera motion vector trajectory is determined based on theglobal motion estimates computed at operation 230. This trajectory maybe considered the result of applying a low pass filter to the estimatedcamera's motion.

At operation 240 motion instability within the video is monitored and/ordetected via analysis of video frames. In one exemplary embodiment,operation 240 entails a computation of motion jitter values for one ormore frames. As an example, motion jitter values may be determined for agiven frame by subtracting out the smoothed camera trajectory computedat operation 235 from the actual position associated with the videoframe. As shown in FIG. 2, method 201 either proceeds to frame warpingoperation 290 or bypasses application of such image stabilization asconditioned on the motion jitter value(s). While any frame warpingtechnique may be applied at frame warping operation 290, it is notedthan many techniques rely, at least in part, on motion jitter values todetermine the proper image filter. Whereas, stabilization typicallyutilizes motion jitter values on a frame-by-frame basis, in embodimentsherein motion instability detection is facilitated by buffering motionjitter values (e.g., in jitter value buffer 145) over a larger number offrames (e.g., many tens of frames). In embodiments therefore, motionjitter values are employed for assessing the need for videostabilization and/or conditioning application of video stabilization, aswell as for actually stabilizing the frame. Consequently, adaptivemotion instability detection may require minimum addition computationalcomplexity beyond that of basic non-adaptive and/or non-conditionalvideo stabilization. Method 201 completes with storing output videoframes (e.g., stabilized, and/or unstabilized) to a (video) memory atoperation 295.

FIG. 3 is a flow diagram illustrating a video processing method 301 withadaptive toggling of video stabilization based on motion instabilitystates, in accordance with an embodiment. In an embodiment, method 301is performed by adaptive video stabilization system 118. Method 301 isone exemplary embodiment of motion instability detection operation 240and in a further embodiment is performed by adaptive motion instabilitydetector 140. Method 301 begins with receiving video frames at operation305. From one or more of the recited video frames, an initial state ofmotion instability is determined at operation 330. In embodiments, adetermination of the initial motion instability state is based on asingle analysis window spanning a predetermined number of receivedframes. Within this window, a motion instability/stability score iscomputed and that score may be thresholded to determine the initialinstability state. At operation 334, received video frames are thenmonitored for a transition in the motion instability state by comparinga first level of instability associated with a first subset of thereceived frames with a second level of instability associated with asecond subset of the received frames. In other words, at least twomotion instability assessments are made based on two windows spanningsome number of frames within different portions of the received video.At operation 340, video stabilization is toggled based on both theinitial instability state and on instability state transitions. Theinitial motion instability state may be fixed or “locked” based on theinstability/stability score determined at operation 330. Imageprocessing-based video stabilization is either applied, or not, toreceived video frames until any instability state transition is detectedat operation 335, at which point image processing-based videostabilization is either discontinued or introduced. Transitions are thencontinually monitored via operation 335 over all the received videoframes with application of video stabilization at operation 340 trackingdetected video motion instability transitions. For example, if aninstability score determined at operation 330 satisfies a threshold andthe initial instability state is therefore deemed unstable, or “shaky,”video stabilization is toggled at operation 340 to apply imagestabilization processing to the received frames until toggled off atoperation 340 in response to detecting any instability state transitionat operation 335. If instead the instability score determined atoperation 330 fails to satisfy a threshold and the video frames deemedstable, video stabilization is toggled such that received frames bypassimage stabilization processing until toggled again in response to anyinstability state transition detected at operation 335.

FIG. 4 is a flow diagram illustrating a method 401 of determining aninitial video motion instability state and monitoring for transitions inthe instability state, in accordance with an embodiment. The method 401illustrates one exemplary embodiment of operations 330 and 335illustrated in FIG. 3. Method 401 begins with receiving input videoframes at operation 205. At operation 410 jitter values are computed,for example, based on residual motion vectors determined from globalmotion estimates and low-pass filtered camera trajectories. Such jittervalues are buffered as they are computed with each incoming frame. Witheach additional input frame, the number of jitter values buffered iscompared against a predetermined jitter value count threshold (m) toconditionally branch between a first algorithm for determining aninitial video motion instability state and a second algorithm formonitoring video motion instability state transitions. The predeterminedjitter value count threshold m may be a programmable parameter, forexample selectable by an application layer and configurable through avideo processor driver. A smaller jitter value count threshold m mayallow more rapid application of video motion stabilization (i.e.,stabilization at a lower frame number) while a larger number may improveconfidence in the initial state determination. Exemplary jitter valuecount thresholds are between 40 and 200 frames.

If the jitter value count threshold m is not yet satisfied (e.g., countis less than m), method 401 proceeds to operation 430, where the initiallevel of motion instability is determined. This initial level is thenthresholded to determine the initial instability state. Where theinitial level satisfies the threshold, the initial video instabilitystate (State₀) is deemed “shaky.” Where the initial level fails tosatisfy the threshold, State₀ is deemed “not shaky.” The predeterminedinitial state threshold may again be a programmable parameter, forexample selectable by an application layer and configurable through avideo processor driver. In embodiments, operation 430 entailsdetermining an instability score for at least some of the firstplurality of video frames that are collected up to reaching the jittervalue count threshold m. Each instability score is indicative of motionjitter values associated with a predetermined number of prior videoframes (n). The motion instability score may be determined at operation430 over some number of subsets with the m jitter values. Depending onthe number of jitter values in the subset relative to the jitter valuecount threshold m, any number of instability scores may be generated atoperation 430. Where m and n are equal for example, one instabilityscore determined at operation 430 may be thresholded to determine theinitial instability state. In the event that m is larger than n, aplurality of instability scores determined at operation 430 may bereduced to a statistic indicative all the scores accumulated over themotion jitter value count threshold m. This statistic may then becompared against the instability level threshold to determine theinitial instability state.

An instability score may be determined through application of variousalgorithms at operation 430. In an exemplary embodiment, a set ofinstability motion jitter values is tested for normality to determinelevels of video motion. A premise of such a test is that “shaky” motionjitter within the x-dimension or y-dimension motion vector components israndomly distributed. As such, the distribution of jitter valuesassociated with a number of prior video frames may be scored based onthe extent to which they follow a Gaussian distribution. A set of jittervalues more normally distributed, for example, may be assigned a higherinstability score while a set of jitter values less normally distributedmay be assigned a lower instability score. Video stabilization may thenbe applied selectively to frames associated with a sufficiently highinstability score (i.e., in a “shaky” state).

FIG. 5 is a data flow diagram illustrating processing of video todetermine an initial video motion instability state, in accordance withan embodiment. The techniques illustrated in FIG. 5 may be employed in aspecific embodiment of operations 330 and 340 in FIG. 3. As shown inFIG. 5, jitter vector J₁ through J_(m) are associated with a last mframes received as input video frames at operation 410. Hence, uponreceiving video frame i, a jitter vector J is computed for each of nconsecutive video frames associated with jitter vector J_(i−n+1) throughJ_(i). Notably, for each jitter vector J denoted in FIG. 5, two jittervalues corresponding to x and y components of a motion vector may becomputed (e.g., J_(x,1) and J_(y,1) through J_(x,m+1) and J_(y,m+1)). Atoperation 515, instability scores are determined. In the exemplaryembodiment, an instability score is determined for each video frameusing a statistical method to compare the jitter values' probabilitydistribution of the previous n consecutive frames. In one suchembodiment, each instability score is based on a hypothesis testcomparing a Gaussian distribution with a distribution of the motionjitter values associated with the n preceding frames. An exemplaryhypothesis test (HT) is Lilliefor's HT, which is suitable where a meanand variance are not known. For such an embodiment, each instabilityscore may be 0 or 1, for a HT failure and HT pass, respectively. Hence,a first instability score is associated with a frame havingcorresponding jitter value J_(n). Every frame/jitter value thereafter isassociated with an instability score derived from the proceeding nframes/jitter values until at least frame m is received. In theexemplary embodiment where the same instability-scoring algorithm isapplied as a basis for detecting a transition in the motion jitter,every received frame in a given video is scored over a moving widowencompassing the previous n consecutive frames. Each instability scoremay be representative of both jitter value components (e.g.,J_(x,i−n+1), J_(y,i−n+1)), for example taking a score of the most motionunstable component. Alternatively, separate x and y componentinstability scores may be determined.

As previously described, instability scores may be accumulated up to thejitter value count threshold m, at which point the accumulationstatistic may be thresholded to determine an initial instability statefrom which subsequent transitions will be detected. In the exemplaryembodiment illustrated in FIG. 5, a cumulative average instability scoreis generated for each jitter value between at least J_(n) and J_(m). Thecumulative average instability score may again be separated by x and ycomponents, or reduced into a worst-case instability score, etc. Atoperation 540, image processing-based motion stabilization is applied inresponse to the instability score statistic satisfying a predeterminethreshold, which may be user programmable through an application layer.The initial instability state, and resulting application of imagestabilization, may be locked at the state determined based on acumulative average instability score at frame m. In one such embodiment,until frame m is received, motion stabilization is applied on aper-frame basis as a function of the instability score(s) correspondingto the received frame. For example, a video frame i is deemed “shaky” ifthe cumulative average Gaussian distribution comparison results from afirst frame (e.g., associated with jitter value J_(n)) to the currentframe (e.g., associated with jitter value J_(i)) satisfies apredetermined level of instability (e.g. >0.75). Otherwise the currentframe is considered a stable frame. Frame stabilization may therefore betoggled “on” as early as the frame associated with jitter value J_(n).In an alternate embodiment, no motion stabilization is applied to anyframe until frame m is received and the initial instability state isdefined based on the cumulative average instability score (e.g., 0.75 atframe m). In either embodiment, accumulation of instability scoreswithin the first m frames allows for increased confidence in thedetermined instability state as jitter count threshold m becomes larger.

Returning to FIG. 4, if the jitter value count threshold m is satisfied,method 401 proceeds to operation 433, where instability statetransitions are monitored. In the exemplary embodiment a change instability level is thresholded. A predetermined state transitionthreshold may again be a programmable parameter, for example selectableby an application layer and configurable through a video processordriver. Where the stability level change satisfies the threshold, thevideo instability state (State_(i)) is toggled to the converse of theprior instability state (State_(i−1)). Where the stability level changefails to satisfy the threshold, the current instability state ismaintained. Noting that instability detection may entail the analysis oftwo components (x,y) of inter-frame motion vectors, instability statetransitions may likewise take a number of forms where the before (first)or after (second) instability states may have instability in either orboth the x and y components. FIG. 6 is a state diagram illustratingstate transitions monitored by an adaptive motion instability detectionsystem, in accordance with an embodiment. As depicted motion instabilityis present in each of states 605 (where there is motion shake in onlyx), 610 (where there is motion shake in only y), and 615 (where there ismotion shake both x and y). Transitions from/to any of states 605, 610,and 615 to/from the “not shaky” state 620 is to be detected at operation433 in FIG. 4.

FIG. 7 is a data flow diagram illustrating processing of video tomonitor transitions in video motion instability state, in accordancewith an embodiment. The techniques illustrated in FIG. 7 may be employedas one specific embodiment of operations 335 and 340 in FIG. 3. In FIG.7, the notation introduced in FIG. 5 is maintained. Beginning atoperation 410, jitter values J (e.g., J_(i,x), J_(i,y)) are computed fora plurality of frames as previously described. At operation 715, a firstinstability score is determined for at least some of the plurality ofvideo frames based on their corresponding jitter values. A secondinstability score is determined for at least some other of the pluralityof video frames, again based on their corresponding jitter values. Thispair of score groups may be compared to detect a change in jitter valueprobability distribution across video frames.

For one such an embodiment, at a frame i, 4n−2 jitter values areemployed for instability state transition detection. The 4n−2 jittervalues are equally divided into two groups illustrated as jitter valuesample windows 716 and 717, respectively. In this exemplary embodimentwhere an instability score is determined from n video frames, ninstability scores may be generated for each group of 2n−1 jitter valuesassociated with 2n−1 frames. In a further embodiment, a normality test(e.g., Lilliefor's HT) is performed over n preceding jitter values, togenerate n instability scores indicative of jitter values associatedwith n frames just as for the initial motion instability determinationdescribed elsewhere herein. For example, a HT score (e.g., 0 or 1) maybe determined for each frame associated with jitter value J_(1−4n+3)through J_(i−2n+1) within the jitter sample window 716. Likewise, HTscores may be determined for each frame associated with jitter valueJ_(i−2n+2) through J_(i) within the jitter sample window 717.

The first and second sets of n instability scores are then compared atoperation 718. A statistic for each of the first and second sets ofinstability scores may be employed in the comparison. At operation 740,motion stabilization may be applied in response to a difference betweenthe first statistic (e.g., associated with n first normality tests fromjitter sample window 716) and the second statistic (e.g., associatedwith n second normality tests from jitter sample window 717), satisfyinga predetermined threshold. For example, where the statistic is anaverage of the instability score over n motion jitter values, thecomparison performed at operation 718 may be the absolute differencebetween a summation of the instability score over the n frames withinjitter sample window 716 and a summation of the instability score overthe n frames within jitter sample window 717, divided by n. Since theprobability distributions of the jitter values should change to bemore/less Gaussian with changes in motion instability, the absolutedifference between the instability scores accumulated over the pair ofjitter sample windows 716, 717 will deviate when the windows straddlethe instability transition point (e.g., between jitter values J_(i−2n+1)and J_(i−2n+2) in FIG. 7).

If the absolute difference satisfies the threshold at operation 740(e.g., is of sufficiently large magnitude), an instability statetransition is deemed to have occurred and the application of motionstabilization to video frames subsequent to frame i is toggled. Forexample, if motion stabilization was “off” as of frame i, it is turned“on” at frame i and remains on until another instability statetransition is subsequently detected using the same algorithm.Alternatively, if motion stabilization was “off” as of frame i, it isturned “on” at frame i and remains on until another instability statetransition is subsequently detected. If the absolute difference fails tosatisfy the threshold at operation 740, the instability states over thelast 4n−2 frames are deemed sufficiently similar for the videostabilization applied as of frame i (e.g., stabilization either “on” or“off”) to be maintained at frame i and thereafter until an instabilitystate transition is subsequently detected.

FIG. 8 is an illustrative diagram of an exemplary system 800, inaccordance with embodiments. System 800 may implement all or a subset ofthe various functional blocks depicted in FIG. 2. For example, in oneembodiment a graphics processor 815 implements a graphics processingsystem that includes the adaptive motion instability detector 140 as wasintroduced in FIG. 1B, for example having one or more of the featuresdescribed elsewhere herein to perform any of the method described in thecontext of FIGS. 2-7. In one specific exemplary embodiment, graphicsprocessor 815 includes fixed-function and/or programmable logiccircuitry within at least one execution unit (EU), or shader core, toperform adaptive video motion instability detection, and/or selectivemotion stabilization of input video in response to detecting motioninstability, or transitions thereof, in input video. System 800 may be amobile device although system 800 is not limited to this context. Forexample, system 800 may be incorporated into a laptop computer,ultra-laptop computer, tablet, touch pad, portable computer, handheldcomputer, palmtop computer, cellular telephone, smart device (e.g.,smart phone, smart tablet or mobile television), mobile internet device(MID), messaging device, data communication device, and so forth. System800 may also be an infrastructure device. For example, system 800 may beincorporated into a large format television, set-top box, desktopcomputer, or other home or commercial network device.

In embodiments, platform 802 may include any combination of a chipset805, processor 810, memory 812, storage 811, graphics processor 815,applications 816 and/or radio 818. Chipset 805 may provideintercommunication among processor 810, memory 812, storage 811,graphics processor 815, applications 816, or radio 818. For example,chipset 805 may include a storage adapter (not depicted) capable ofproviding intercommunication with storage 811.

In embodiments, platform 802 may include any combination of a chipset805, processor 810, memory 812, storage 814, graphics processor 815,applications 816 and/or radio 818. Chipset 805 may provideintercommunication among processor 810, memory 812, storage 814,graphics processor 815, applications 816, or radio 818. For example,chipset 805 may include a storage adapter (not depicted) capable ofproviding intercommunication with storage 814.

Processor 810 may be implemented as one or more Complex Instruction SetComputer (CISC) or Reduced Instruction Set Computer (RISC) processors;x86 instruction set compatible processors, multi-core, or any othermicroprocessor or central processing unit (CPU). In embodiments,processor 810 may be a multi-core processor(s), multi-core mobileprocessor(s), and so forth.

Memory 812 may be implemented as a volatile memory device such as, butnot limited to, a Random Access Memory (RAM), Dynamic Random AccessMemory (DRAM), or Static RAM (SRAM).

Storage 811 may be implemented as a non-volatile storage device such as,but not limited to, a magnetic disk drive, optical disk drive, tapedrive, an internal storage device, an attached storage device, flashmemory, battery backed-up SDRAM (synchronous DRAM), and/or a networkaccessible storage device. In embodiments, storage 811 may includetechnology to increase the storage performance enhanced protection forvaluable digital media when multiple hard drives are included, forexample.

Graphics processor 815 may perform processing of images such as still orvideo media data for display, or perform general computing functions ina highly parallel manner. Graphics processor 815 may include one or moreGPU, or visual processing unit (VPU), for example. An analog or digitalinterface may be used to communicatively couple graphics processor 815and display 820. For example, the interface may be any of aHigh-Definition Multimedia Interface, Display Port, wireless HDMI,and/or wireless HD compliant techniques. Graphics processor 815 may beintegrated with central processor 810 onto a single chip (i.e., SoC) asa graphics core or provided as part of chipset 805. In someimplementations, graphics processor 815 may be a stand-alone cardcommunicatively coupled to chipset 805. In various exemplaryembodiments, graphics processor 815 and/or central processor 810 invokesor otherwise implements video motion instability mediation processes.Graphics processor 815 includes functionality to perform adaptive motioninstability detection methods upon which video motion instabilitymediation processes may be predicated, for example as describedelsewhere herein.

The video motion instability mediation processes predicated uponadaptive motion instability detection as described herein may beimplemented in various hardware architectures, cell designs, or “IPcores.” As still another embodiment, the methods and functions describedherein in the context of graphics processor may be extended to ageneral-purpose processor, including a multi-core processor. In furtherembodiments, the methods and functions may be implemented in apurpose-built consumer electronics device, such as a game consoleprocessor.

Radio 818 may include one or more radios capable of transmitting andreceiving signals using various suitable wireless communicationstechniques. Such techniques may involve communications across one ormore wireless networks. Example wireless networks include (but are notlimited to) wireless local area networks (WLANs), wireless personal areanetworks (WPANs), wireless metropolitan area network (WMANs), cellularnetworks, and satellite networks. In communicating across such networks,radio 718 may operate in accordance with one or more applicablestandards in any version.

In embodiments, HID 820 may include any television type monitor ordisplay. HID 820 may include, for example, a computer display screen,touch screen display, video monitor, television-like device, and/or atelevision. HID 820 may be digital and/or analog. In embodiments, HID820 may be a holographic display. Also, HID 820 may be a transparentsurface that may receive a visual projection. Such projections mayconvey various forms of information, images, and/or objects. Forexample, such projections may be a visual overlay for a mobile augmentedreality (MAR) application. Under the control of one or more softwareapplications 816, platform 802 may display user interface 822 on HID820.

In embodiments, platform 802 may receive control signals from navigationcontroller 850 having one or more navigation features. The navigationfeatures of controller 850 may be used to interact with user interface822, for example. In embodiments, navigation controller 850 may be apointing device that may be a computer hardware component (specifically,a human interface device) that allows a user to input spatial (e.g.,continuous and multi-dimensional) data into a computer. Many systemssuch as graphical user interfaces (GUI), and televisions and monitorsallow the user to control and provide data to the computer or televisionusing physical gestures.

Movements of the navigation features of controller 850 may be replicatedon a display (e.g., HID 820) by movements of a pointer, cursor, focusring, or other visual indicators displayed on the display. For example,under the control of software applications 816, the navigation featureslocated on navigation controller 850 may be mapped to virtual navigationfeatures displayed on user interface 822, for example. In embodiments,controller 850 may not be a separate component but may be integratedinto platform 802 and/or HID 820. The present disclosure, however, isnot limited to the elements or in the context shown or described herein.

In embodiments, system 800 may be implemented as a wireless system, awired system, or a combination of both. When implemented as a wirelesssystem, system 800 may include components and interfaces suitable forcommunicating over a wireless shared media, such as one or moreantennas, transmitters, receivers, transceivers, amplifiers, filters,control logic, and so forth. An example of wireless shared media mayinclude portions of a wireless spectrum, such as the RF spectrum and soforth. When implemented as a wired system, system 800 may includecomponents and interfaces suitable for communicating over wiredcommunications media, such as input/output (I/O) adapters, physicalconnectors to connect the I/O adapter with a corresponding wiredcommunications medium, a network interface card (MC), disc controller,video controller, audio controller, and the like. Examples of wiredcommunications media may include a wire, cable, metal leads, printedcircuit board (PCB), backplane, switch fabric, semiconductor material,twisted-pair wire, co-axial cable, fiber optics, and so forth.

As described above, system 800 may be embodied in varying physicalstyles or form factors. FIG. 9 illustrates embodiments of a small formfactor device 990 in which system 1000 may be embodied. In embodiments,for example, device 990 may be implemented as a mobile computing devicehaving wireless capabilities. A mobile computing device may refer to anydevice having a processing system and a mobile power source or supply,such as one or more batteries, for example.

Examples of a mobile computing device may include a personal computer(PC), laptop computer, ultra-laptop computer, tablet, touch pad,portable computer, handheld computer, palmtop computer, personal digitalassistant (PDA), cellular telephone, combination cellular telephone/PDA,television, smart device (e.g., smartphone, tablet or smart television),mobile internet device (MID), messaging device, data communicationdevice, and so forth.

Examples of a mobile computing device also may include computers and/ormedia capture/transmission devices configured to be worn by a person,such as a wrist computer, finger computer, ring computer, eyeglasscomputer, belt-clip computer, arm-band computer, shoe computers,clothing computers, and other wearable computers. In variousembodiments, for example, a mobile computing device may be implementedas a smart phone capable of executing computer applications, as well asvoice communications and/or data communications. Although someembodiments may be described with a mobile computing device implementedas a smart phone by way of example, it may be appreciated that otherembodiments may be implemented using other wireless mobile computingdevices as well. The embodiments are not limited in this context.

As shown in FIG. 9, device 990 may include a housing 902, a display 907,an input/output (I/O) device 906, and an antenna 908. Device 990 alsomay include navigation features 912. Display 907 may include anysuitable display unit for displaying information 910 appropriate for amobile computing device. I/O device 1206 may include any suitable I/Odevice for entering information 910 into a mobile computing device.Examples for I/O device 906 may include an alphanumeric keyboard, anumeric keypad, a touch pad, input keys, buttons, switches, rockerswitches, microphones, speakers, voice recognition device and software,and so forth. Information 910 also may be entered into device 990 by wayof microphone (not shown), or may be digitized by a voice recognitiondevice. Embodiments are not limited in this context.

Embodiments described herein may be implemented using hardware elements,software elements, or a combination of both. Examples of hardwareelements or modules include: processors, microprocessors, circuitry,circuit elements (e.g., transistors, resistors, capacitors, inductors,and so forth), integrated circuits, application specific integratedcircuits (ASIC), programmable logic devices (PLD), digital signalprocessors (DSP), field programmable gate array (FPGA), logic gates,registers, semiconductor device, chips, microchips, chip sets, and soforth. Examples of software elements or modules include: applications,computer programs, application programs, system programs, machineprograms, operating system software, middleware, firmware, routines,subroutines, functions, methods, procedures, software interfaces,application programming interfaces (API), instruction sets, computingcode, computer code, code segments, computer code segments, data words,values, symbols, or any combination thereof. Determining whether anembodiment is implemented using hardware elements and/or softwareelements may vary in accordance with any number of factors consideredfor the choice of design, such as, but not limited to: desiredcomputational rate, power levels, heat tolerances, processing cyclebudget, input data rates, output data rates, memory resources, data busspeeds and other design or performance constraints.

One or more aspects of at least one embodiment may be implemented byrepresentative instructions stored on a machine-readable storage medium.Such instructions may reside, completely or at least partially, within amain memory and/or within a processor during execution thereof by themachine, the main memory and the processor portions storing theinstructions then also constituting a machine-readable storage media.Programmable logic circuitry may have registers, state machines, etc.configured by the processor implementing the computer readable media.Such logic circuitry, as programmed, may then be understood to have beenphysically transformed into a system falling within the scope of theembodiments described herein. Instructions representing various logicwithin the processor, which when read by a machine may also cause themachine to fabricate logic adhering to the architectures describedherein and/or to perform the techniques described herein. Suchrepresentations, known as cell designs, or IP cores, may be stored on atangible, machine-readable medium and supplied to various customers ormanufacturing facilities to load into the fabrication machines thatactually make the logic or processor.

While certain features set forth herein have been described withreference to embodiments, this description is not intended to beconstrued in a limiting sense. Hence, various modifications of theimplementations described herein, as well as other implementations,which are apparent to persons skilled in the art to which the presentdisclosure pertains are deemed to be within the spirit and scope of thepresent disclosure.

The following examples pertain to particular exemplary embodiments.

In first embodiments, a computer-implemented video processing methodincludes receiving video frames, determining an initial state of motioninstability associated with a plurality of the frames, and monitoringfor a motion instability state transition by comparing a first level ofinstability associated with a first plurality of the frames with asecond level of instability associated with a second plurality of theframes. The method further includes toggling image stabilization ofreceived video frames based on the initial instability state, and basedon an instability state transition, and storing the stabilized ornon-stabilized video frames to a memory.

In furtherance of the first embodiments, toggling the imagestabilization further includes applying motion stabilization to receivedvideo frames in response to an initial level of instability satisfying afirst threshold, and until the change in the level of instabilitysatisfies a second threshold. Alternatively, toggling the imagestabilization further includes bypassing motion stabilization inresponse to the initial level of instability failing to satisfy thefirst threshold, and until the change in the level of instabilitysatisfies the second threshold.

In furtherance of the first embodiments, determining the initialinstability state further includes computing a motion jitter value foreach of a first plurality of video frames. Determining the initialinstability state further includes determining a first instability scorefor at least some of the first plurality of video frames, each scoreindicative of motion jitter values associated with a predeterminednumber of prior video frames. Determining the initial instability statefurther includes determining a statistic indicative of first instabilityscores accumulated over the predetermined number of first motion jittervalues. Determining the initial instability state further includescomparing the statistic against a threshold.

In furtherance of the first embodiments, determining the initialinstability state further includes computing a motion jitter value foreach of a first plurality of video frames. Determining the initialinstability state further includes determining a first instability scorefor at least some of the first plurality of video frames, each scorebased on a comparison between the Gaussian distribution and adistribution of the motion jitter values associated with a predeterminednumber of prior video frames. Determining the initial instability statefurther includes determining an average of the first instability scoresover the predetermined number of first motion jitter values. Determiningthe initial instability state further includes comparing the averageagainst a threshold.

In furtherance of the first embodiments, determining the initialinstability state further includes computing first motion jitter valuesfor video frames until a predetermined number of first motion jittervalues have been computed. Determining the initial instability statefurther includes determining a first instability score indicative of oneor more of the first motion jitter values. Determining the initialinstability state further includes determining a first statisticindicative of the first instability scores accumulated over thepredetermined number of first motion jitter values. Monitoring for themotion instability state transition further includes computing secondmotion jitter values for video frames after the predetermined number ofjitter values has been computed. Monitoring for the motion instabilitystate transition further includes thresholding a comparison between oneor more of the first instability scores and one or more secondinstability scores indicative of the second motion jitter values.

In furtherance of the first embodiments, determining levels of motioninstability further includes computing a motion jitter value for each ofa plurality of video frames. Determining levels of motion instabilityfurther includes testing the motion jitter values for normality.Toggling the image stabilization based on the initial instability statefurther includes applying motion stabilization to received video framesin response to the motion jitter values satisfying the normality test.Toggling the image stabilization based on the initial instability statefurther includes bypassing motion stabilization in response to themotion jitter values failing the normality test.

In furtherance of the first embodiments, determining levels of motioninstability further includes computing a motion jitter value for each ofa plurality of video frames. Determining levels of motion instabilityfurther includes testing the motion jitter values for normality.Toggling the image stabilization based on the change in the level ofinstability further includes modifying the application of motionstabilization to received video frames in response to a differencebetween a first statistic, associated with a first plurality ofnormality tests, and a second statistic, associated with a secondplurality of normality tests, satisfying a threshold. Toggling the imagestabilization based on the change in the level of instability furtherincludes maintaining the application of motion stabilization in responseto the difference between the first and second statistic failing tosatisfy the threshold.

In furtherance of the first embodiments, monitoring for the motioninstability state transition further includes computing a motion jittervalue for each of a first plurality of video frames. Monitoring for themotion instability state transition further includes determining a firstinstability score for at least some of the first plurality of videoframes, each score indicative of motion jitter values associated with apredetermined number of prior video frames. Monitoring for the motioninstability state transition further includes determining a firststatistic of the first instability scores. Monitoring for the motioninstability state transition further includes computing a motion jittervalue for each of a second plurality of video frames. Monitoring for themotion instability state transition further includes determining asecond instability score for at least some of the second plurality ofvideo frames, each score indicative of motion jitter values associatedwith the predetermined number of prior video frames. Monitoring for themotion instability state transition further includes determining asecond statistic of the second instability scores. Monitoring for themotion instability state transition further includes comparing athreshold to a difference between the first and second statistic.

In furtherance of the first embodiments, receiving video frames furthercomprises receiving a video frame i. Determining the initial motioninstability state associated with a plurality of the frames furtherincludes computing a jitter value for each of n consecutive video framesinclusive of jitter values J_(i) and J_(n). Determining the initialmotion instability state associated with a plurality of the framesfurther includes determining an instability score for each of the nvideo frames, wherein each of the instability scores is based on ahypothesis test comparing a Gaussian distribution with a distribution ofthe motion jitter values associated with n frames. Determining theinitial motion instability state associated with a plurality of theframes further includes averaging the instability scores over the nvideo frames. Toggling image stabilization of received video framesbased on the initial instability state further includes toggling motionstabilization to video frames received after frame i in response to theaveraged instability score satisfying a first threshold.

In furtherance of the embodiment described above, monitoring for themotion instability state transition further includes computing a motionjitter value for each of 4n−2 consecutive video frames inclusive of 2n−1frames associated with jitter values J_(i) through J_(i−2n+2) and 2n−1frames associated with jitter values J_(i−2n+1) and J_(i−4n+3).Monitoring for the motion instability state transition further includesdetermining a first instability score for each of n video frames endingwith video frame associated with jitter value J_(i−2n+1), wherein eachof the first instability scores is based on a hypothesis test comparinga Gaussian distribution with a distribution of the motion jitter valuesassociated with n frames. Monitoring for the motion instability statetransition further includes averaging the first instability scores overn video frames. Monitoring for the motion instability state transitionfurther includes determining a second instability score for each of nvideo frames ending with video frame i, wherein each of the secondinstability scores is based on the hypothesis test. Monitoring for themotion instability state transition further includes averaging thesecond instability scores over n video frames, and comparing a secondthreshold to a difference between the first and second averagedinstability score. Toggling image stabilization of received video framesbased on the change in the level of instability further includesmodifying the application of motion stabilization to received videoframes in response to the difference between the first and secondaveraged instability score satisfying the second threshold. Togglingimage stabilization of received video frames based on the change in thelevel of instability further includes maintaining an application ofmotion stabilization in response to the difference between the first andsecond averaged instability score failing to satisfy the secondthreshold.

In second embodiments, a processor includes logic circuitry to receivevideo frames. The processor includes motion instability detection logiccircuitry to determine an initial motion instability state associatedwith a plurality of the frames. The processor includes motioninstability detection logic circuitry to monitor for a motioninstability state transition by comparing a first level of instabilityassociated with a first plurality of the frames with a second level ofinstability associated with a second plurality of the frames. Theprocessor includes motion instability detection logic circuitry totoggle image stabilization of video frames based on the initial motioninstability state, and based on a motion instability state transition.

In furtherance of the second embodiments, the motion instabilitydetection logic circuitry is further to apply motion stabilization toreceived video frames in response to an initial level of instabilitysatisfying a first threshold, and until the change in the level ofinstability satisfies a second threshold. The motion instabilitydetection logic circuitry is further to bypass motion stabilization inresponse to the initial level of instability failing to satisfy thefirst threshold, and until the change in the level of instabilitysatisfies the second threshold.

In furtherance of the second embodiments, the motion instabilitydetection logic circuitry is to compute first motion jitter values forvideo frames until a predetermined number of first motion jitter valueshave been computed. The motion instability detection logic circuitry isto determine a first instability score indicative of one or more of thefirst motion jitter values. The motion instability detection logiccircuitry is to determine a statistic indicative of first instabilityscores accumulated over the predetermined number of first motion jittervalues. The motion instability detection logic circuitry is to togglethe image stabilization based on the statistic. The motion instabilitydetection logic circuitry is further to compute second motion jittervalues for video frames after the predetermined number of jitter valueshas been computed. The motion instability detection logic circuitry isfurther to threshold a comparison between the first instability scoreand a second instability score indicative of one or more of the secondmotion jitter values, and toggle the image stabilization based on thecomparison.

In furtherance of the second embodiments, the motion instabilitydetection logic circuitry is to compute a motion jitter value for eachof a plurality of video frames. The motion instability detection logiccircuitry is to test the motion jitter values for normality. The motioninstability detection logic circuitry is to perform motion stabilizationon received video frames in response to the motion jitter valuessatisfying the normality test. The motion instability detection logiccircuitry is to bypass motion stabilization in response to the motionjitter values failing the normality test.

In furtherance of the second embodiments, the motion instabilitydetection logic circuitry is to compute a motion jitter value for eachof a plurality of video frames. The motion instability detection logiccircuitry is to test the motion jitter values for normality. The motioninstability detection logic circuitry is to modify application of motionstabilization to received video frames in response to a differencebetween a first statistic associated with a first plurality of normalitytests, and a second statistic, associated with a second plurality ofnormality tests, satisfying a threshold. The motion instabilitydetection logic circuitry is to maintain application of motionstabilization in response to the difference between the first and secondstatistic failing to satisfy the threshold.

In furtherance of the second embodiments, the motion instabilitydetection logic circuitry is to compute a motion jitter value for eachof a first plurality of video frames. The motion instability detectionlogic circuitry is to determine a first instability score for at leastsome of the first plurality of video frames, each score based on acomparison between the Gaussian distribution and a distribution of themotion jitter values associated with a predetermined number of priorvideo frames. The motion instability detection logic circuitry is todetermine an average of the first instability scores. The motioninstability detection logic circuitry is to compare the average of thefirst instability scores against a threshold.

In third embodiments, a video processing system includes one or moreprocessor to receive video frames. The one or more processor is furtherto determine an initial motion instability state associated with aplurality of the frames. The one or more processor is further to monitorfor a motion instability state transition by comparing a first level ofinstability associated with a first plurality of the frames to a secondlevel of instability associated with a second plurality of the frames.The one or more processor is further to toggle image stabilization ofreceived video frames based on the initial instability state, and basedon an instability state transition. The video processing system furtherincludes a memory coupled to the one or more processor to store thestabilized or non-stabilized video frames.

In furtherance of the third embodiments, the video processing systemfurther includes a display device coupled to the memory to display thestabilized or non-stabilized video frames.

In one or more fourth embodiment, one or more computer-readable storagemedium has instructions stored thereon, which when executed by aprocessor, cause the processor to perform a method including determiningan initial motion instability state associated with a plurality of videoframes. The instructions further cause the processor to perform themethod including monitoring for a change in the motion instability bycomparing a first level of instability associated with a first pluralityof the frames with a second level of instability associated with asecond plurality of the frames. The instructions further cause theprocessor to perform the method including toggling image stabilizationof received video frames based on the initial instability state, andbased on a change in the instability state. The instructions furthercause the processor to perform the method including storing thestabilized or non-stabilized video frames to a memory.

In furtherance of the fourth embodiments, one or more computer-readablestorage medium further includes instructions to cause the processor toperform a method, wherein determining the initial instability statefurther includes computing first motion jitter values for video framesuntil a predetermined number of first motion jitter values have beencomputed. The instructions further cause the processor to perform themethod including determining a first instability score indicative of oneor more of the first motion jitter values. The instructions furthercause the processor to perform the method including determining a firststatistic indicative of the first instability scores accumulated overthe predetermined number of first motion jitter values. The instructionsfurther cause the processor to perform the method including monitoringfor the motion instability state transition by computing second motionjitter values for video frames after the predetermined number of jittervalues have been computed, and thresholding a comparison between one ormore of the first instability scores and one or more second instabilityscores indicative of the second motion jitter values.

It will be recognized that the embodiments are not limited to theexemplary embodiments so described, but can be practiced withmodification and alteration without departing from the scope of theappended claims. For example, the above embodiments may include specificcombination of features. However, the above embodiments are not limitedin this regard and, in embodiments, the above embodiments may includethe undertaking only a subset of such features, undertaking a differentorder of such features, undertaking a different combination of suchfeatures, and/or undertaking additional features than those featuresexplicitly listed. Scope should, therefore, be determined with referenceto the appended claims, along with the full scope of equivalents towhich such claims are entitled.

What is claimed is:
 1. A computer-implemented video processing method,comprising: receiving video frames; determining an initial state ofmotion instability associated with a plurality of the frames; monitoringfor a motion instability state transition by comparing a first level ofinstability associated with a first plurality of the frames with asecond level of instability associated with a second plurality of theframes; toggling image stabilization of received video frames based onthe initial state of motion instability, and based on the motioninstability state transition monitoring, wherein the toggling furthercomprises: applying motion stabilization to received video frames inresponse to an initial level of instability satisfying a firstthreshold, and until a change between the first and second levels ofinstability satisfies a second threshold; or bypassing motionstabilization in response to the initial level of instability failing tosatisfy the first threshold, and until a change between the first andsecond levels of instability satisfies the second threshold; and storingstabilized or non-stabilized video frames to a memory.
 2. The method ofclaim 1, wherein determining the initial instability state furthercomprises: computing a motion jitter value for each of the firstplurality of video frames; determining a first instability score for atleast some of the first plurality of video frames, each score indicativeof motion jitter values associated with a predetermined number of priorvideo frames; determining a statistic indicative of first instabilityscores accumulated over the predetermined number of first motion jittervalues; and comparing the statistic against a threshold.
 3. The methodof claim 1, wherein determining the initial instability state furthercomprises: computing a motion jitter value for each of the firstplurality of video frames; determining a first instability score for atleast some of the first plurality of video frames, each score based on acomparison between the Gaussian distribution and a distribution of themotion jitter values associated with a predetermined number of priorvideo frames; determining an average of the first instability scoresover the predetermined number of first motion jitter values; andcomparing the average against a threshold.
 4. The method of claim 1,wherein: determining the initial instability state further comprises:computing first motion jitter values for video frames until apredetermined number of first motion jitter values have been computed;determining a first instability score indicative of one or more of thefirst motion jitter values; and determining a first statistic indicativeof the first instability scores accumulated over the predeterminednumber of first motion jitter values; and monitoring for the motioninstability state transition further comprises: computing second motionjitter values for the second plurality of video frames; and thresholdinga comparison between one or more of the first instability scores and oneor more second instability scores indicative of the second motion jittervalues.
 5. The method of claim 1, further comprising determining thefirst and second levels of motion instability by computing a motionjitter value for each frame of the first and second plurality of videoframes; and testing the motion jitter values for normality; and whereintoggling the image stabilization based on the initial instability statefurther comprises: applying motion stabilization to received videoframes in response to the motion jitter values satisfying the normalitytest; and bypassing motion stabilization in response to the motionjitter values failing the normality test.
 6. The method of claim 1,further comprising determining the first and second levels of motioninstability by computing a motion jitter value for each frame of thefirst and second plurality of video frames; and testing the motionjitter values for normality; and wherein toggling the imagestabilization based on the change in the level of instability furthercomprises: modifying the application of motion stabilization to receivedvideo frames in response to a difference between a first statistic,associated with a first plurality of normality tests determined for thefirst plurality of frames, and a second statistic, associated with asecond plurality of normality tests determined for the second pluralityof frames, satisfying a threshold; and maintaining the application ofmotion stabilization in response to the difference between the first andsecond statistic failing to satisfy the threshold.
 7. The method ofclaim 1, wherein monitoring for the motion instability state transitionfurther comprises: computing a motion jitter value for each of the firstplurality of video frames; determining a first instability score for atleast some of the first plurality of video frames, each score indicativeof motion jitter values associated with a predetermined number of priorvideo frames; determining a first statistic of the first instabilityscores; computing a motion jitter value for each of the second pluralityof video frames; determining a second instability score for at leastsome of the second plurality of video frames, each score indicative ofmotion jitter values associated with the predetermined number of priorvideo frames; determining a second statistic of the second instabilityscores; and comparing a threshold to a difference between the first andsecond statistic.
 8. The method of claim 1, wherein: receiving the videoframes further comprises receiving a video frame i; determining theinitial motion instability state associated with a plurality of theframes further comprises: computing a jitter value for each of nconsecutive video frames inclusive of jitter values J_(i) and J_(n);determining an instability score for each of the n video frames, whereineach of the instability scores is based on a hypothesis test comparing aGaussian distribution with a distribution of the motion jitter valuesassociated with n frames; and averaging the instability scores over then video frames; and toggling image stabilization of received videoframes based on the initial instability state further comprises:toggling motion stabilization to video frames received after frame i inresponse to the averaged instability score satisfying a first threshold.9. The method of claim 8, wherein: monitoring for the motion instabilitystate transition further comprises: computing a motion jitter value foreach of 4n−2 consecutive video frames inclusive of 2n−1 framesassociated with jitter values J_(i) through J_(i−2n+2) and 2n−1 framesassociated with jitter values J_(i−2n+1) and J_(i−4n+3); determining afirst instability score for each of n video frames ending with videoframe associated with jitter value J_(i−2n+1), wherein each of the firstinstability scores is based on a hypothesis test comparing a Gaussiandistribution with a distribution of the motion jitter values associatedwith n frames; averaging the first instability scores over n videoframes; determining a second instability score for each of n videoframes ending with video frame i, wherein each of the second instabilityscores is based on the hypothesis test; averaging the second instabilityscores over n video frames; and comparing a second threshold to adifference between the first and second averaged instability score; andtoggling image stabilization of received video frames based on thechange in the level of instability further comprises: modifying theapplication of motion stabilization to received video frames in responseto the difference between the first and second averaged instabilityscore satisfying the second threshold; and maintaining an application ofmotion stabilization in response to the difference between the first andsecond averaged instability score failing to satisfy the secondthreshold.
 10. A processor comprising: logic circuitry to receive videoframes; motion instability detection logic circuitry to: determine aninitial motion instability state associated with a plurality of theframes; monitor for a motion instability state transition by comparing afirst level of instability associated with a first plurality of theframes with a second level of instability associated with a secondplurality of the frames; apply motion stabilization to received videoframes in response to an initial level of instability satisfying a firstthreshold, and until a change between the first and second levels ofinstability satisfies a second threshold; and bypass motionstabilization in response to the initial level of instability failing tosatisfy the first threshold, and until the change between the first andsecond levels of instability satisfies the second threshold.
 11. Theprocessor of claim 10, wherein: the motion instability detection logiccircuitry is to: compute first motion jitter values for the firstplurality of video frames until a predetermined number of first motionjitter values have been computed; determine a first instability scoreindicative of one or more of the first motion jitter values; determine astatistic indicative of first instability scores accumulated over thepredetermined number of first motion jitter values; apply or bypassmotion stabilization for one or more of the video frames based on thestatistic; and the motion instability detection logic circuitry isfurther to: compute second motion jitter values for the second pluralityof video frames; threshold a comparison between the first instabilityscore and a second instability score indicative of one or more of thesecond motion jitter values; and toggle the motion stabilization basedon the comparison.
 12. The processor of claim 10, wherein the motioninstability detection logic circuitry is to: compute a motion jittervalue for each of the first plurality of video frames; test the motionjitter values for normality; perform motion stabilization on receivedvideo frames in response to the motion jitter values satisfying thenormality test; and bypass motion stabilization in response to themotion jitter values failing the normality test.
 13. The processor ofclaim 10, wherein the motion instability detection logic circuitry isto: compute a motion jitter value for each of the first plurality ofvideo frames; test the motion jitter values for normality; and modifyapplication of motion stabilization to received video frames in responseto a difference between a first statistic, associated with a firstplurality of normality tests determined for the first plurality offrames, and a second statistic, associated with a second plurality ofnormality tests determined for the second plurality of frames,satisfying a threshold; and maintain application of motion stabilizationin response to the difference between the first and second statisticfailing to satisfy the threshold.
 14. The processor of claim 10, whereinthe motion instability detection logic circuitry is to: compute a motionjitter value for each of the first plurality of video frames; determinea first instability score for at least some of the first plurality ofvideo frames, each score based on a comparison between the Gaussiandistribution and a distribution of the motion jitter values associatedwith a predetermined number of prior video frames; and determine anaverage of the first instability scores; and compare the average of thefirst instability scores against a threshold.
 15. A video processingsystem, comprising: one or more processor to: receive video frames;compute first motion jitter values for a first plurality of the videoframes until a predetermined number of first motion jitter values havebeen computed; determine a first instability score indicative of one ormore of the first motion jitter values; determine a statistic indicativeof first instability scores accumulated over the predetermined number offirst motion jitter values; apply or bypass motion stabilization for oneor more of the video frames based on the statistic; compute secondmotion jitter values for a second plurality of the video frames;threshold a comparison between the first instability score and a secondinstability score indicative of one or more of the second motion jittervalues; and toggle motion stabilization of received video frames basedon the comparison; and a memory coupled to the one or more processor tostore stabilized or non-stabilized video frames.
 16. The videoprocessing system of claim 15, further comprising a display devicecoupled to the memory, and to display the stabilized or non-stabilizedvideo frames.
 17. One or more non-transitory computer-readable storagemedium having instructions stored thereon, which when executed by aprocessor, cause the processor to perform a method comprising:determining an initial motion instability state associated with aplurality of video frames; monitoring for a change in the motioninstability by comparing a first level of instability associated with afirst plurality of the frames with a second level of instabilityassociated with a second plurality of the frames; toggling imagestabilization of received video frames based on the initial instabilitystate, and based on a change in the instability state, wherein thetoggling further comprises: applying motion stabilization to receivedvideo frames in response to an initial level of instability satisfying afirst threshold, and until a change between the first and second levelsof instability satisfies a second threshold; or bypassing motionstabilization in response to the initial level of instability failing tosatisfy the first threshold, and until a change between the first andsecond levels of instability satisfies the second threshold; and storingstabilized or non-stabilized video frames to a memory.
 18. The one ormore non-transitory computer-readable storage medium of claim 17,further comprising instructions to cause the processor to perform amethod for determining the initial instability state further comprising:computing first motion jitter values for the first plurality of videoframes until a predetermined number of first motion jitter values havebeen computed; determining a first instability score indicative of oneor more of the first motion jitter values; and determining a firststatistic indicative of the first instability scores accumulated overthe predetermined number of first motion jitter values; and furthercomprising instructions to cause the processor to perform a method formonitoring for the motion instability state transition furthercomprising: computing second motion jitter values for the secondplurality of video frames after the predetermined number of jittervalues have been computed; and thresholding a comparison between one ormore of the first instability scores and one or more second instabilityscores indicative of the second motion jitter values.